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SECTION  1 


BACKGROUND 


1.0  INTRODUCTION 

This  report  summarizes  the  activities  of  Project  5720, 

ESD/MITRE  Software  Center  General  Support,  over  fiscal  year  1986. 

1.1  RATIONALE  FOR  THE  SOFTWARE  CENTER 

The  Electronic  Systems  Division  of  the  Air  Force  Systems 
Command  manages  the  acquisition  and  upgrade  of  Command,  Control, 
Communications  and  Intelligence  (C^I)  systems.  Reviews  of 
software-intensive  ESD  programs  frequently  reveal  cost  and  schedule 
overruns.  Software  is  a  significant  contributor  to  these  problems. 

3 

It  is  not  easy  to  provide  software  for  Cl  systems  on  time 
and  with  predictable  cost.  CJI  systems  are  complex,  with  interfaces 
to  people,  equipment,  and  other  systems.  Software  supplies  many  of 
the  capabilities  of  C^I  systems,  and  serves  as  an  integrator  of  its 
diverse  elements.  CJI  requirements  are  difficult  to  define  and 
subject  to  change.  Software  can  be  a  means  for  adaptation  and 
upgrade  of  a  system  throughout  its  life  cycle.  This  means  that 
software  is  on  the  critical  path  to  system  delivery. 

The  Software  Center  was  formed  to  provide  a  focus  of  knowledge 
and  resources  within  ESD  and  MITRE  to  support  ESD  programs  in 
software  acquisition  and  to  address  the  problems  that  arise  due  to 
software  in  system  acquisitions. 


1.2  GOALS  OF  THE  SOFTWARE  CENTER 

The  long-term  goals  of  the  Software  Center  are  to  substantially 
reduce  the  cost  and  schedule  risks  in  software  acquisition  for  ESD 
systems,  and  to  improve  the  responsiveness  of  systems  to  user  needs 
by  providing  useful  capabilities  and  reliable,  maintainable 
software.  To  meet  these  goals,  the  Software  Center  must  be  an 
acknowledged  leader  in  software  acquisition. 
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1.3  GOALS  OF  PROJECT  5720  IN  THE  SOFTWARE  CENTER 


The  relationship  of  Project  5720  to  the  Software  Center  is 
depicted  in  figure  1.  The  goal  of  Project  5720  is  to  help  the  ESD 
Software  Design  Center  (XRS)  to  implement  ESD-wide  improvements  in 
software  acquisition  practices  and  to  facilitate  the  use  of  the  most 
appropriate  software  engineering  technology  in  the  management  and 
the  development  of  software  for  ESDfs  CJ  systems.  Most  of  the  staff 
who  work  on  Project  5720  also  spend  time  providing  direct  support  to 
ESD  programs.  This  facilitates  the  flow  of  information  into  Project 
5720  on  acquisition  problems  which  recur  across  ESD  programs  and 
provides  an  outflow  to  programs  of  information  on  new  technology  and 
new  practices. 


1.4  SCOPE  OF  ACTIVITIES  OF  THE  SOFTWARE  CENTER 


1.4.1  Direct  Acquisition  Support 

A  large  portion  of  the  Center  staff  has  direct  responsibility 
for  software  acquisition  in  specific  ESD  programs.  These 
arrangements  are  usually  made  within  MITRE  between  the  Software 
Center  and  Project  Leaders.  Staff  are  dedicated  to  long-term 
support  in  software-related  activities  ranging  from  pre-RFP 
requirements  specification  through  the  final  testing  phases. 

At  the  last  count,  the  Software  Center  provided  support  to  43 
ESD  programs.  These  programs  cover  a  broad  range  of  mission  areas 
and  stages  in  the  typical  system  life  cycle. 


1.4.2  Short-Term  Support 

The  Software  Center  provides  ad  hoc  support  for  program  reviews 
and  to  supplement  program  office  staff  in  preparing  for  important 
milestones.  Software  Center  personnel  may  participate  in 
independent  review  teams  within  MITRE  or  jointly  with  ESD. 
Assignments  have  included  source  selection,  reviews  of  software 
aspects  of  RFP  packages,  software  sizing  estimates,  and 
troubleshooting  software  problems  during  software  development.  At 
any  one  time,  at  least  four  people  are  likely  to  be  participating  in 
ad  hoc  support  for  reviews  and  still  more  are  providing  short-term 
support  to  ESD  programs. 
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TECHNOLOGY  TRANSITION 


1.4.3  General  Acquisition  Support;  Project  5720 


The  activities  on  Project  5720  provide  general  benefits  to  ESD 
programs  in  all  practices  related  to  software  for  C^I  systems.  The 
project  supports  management  oversight  and  planning  for  the  Software 
Center,  which  numbered  about  120  staff  in  FY86. 


A  role  of  Project  5720  is  to  help  ESD  conform  to  new  software 
acquisition  policy  without  undue  risk.  This  requires  us  to  develop 
rules  of  thumb  for  applying  policies  to  the  requirements  of 
individual  programs.  Project  staff  also  help  to  shape  new  policies 
by  reviewing  draft  policy  and  software  acquisition  guidance  from  the 
DoD  and  Air  Force,  and  by  participating  in  policy-making 
organizations . 


Project  5720  sponsors  the  development,  standardization,  and 
application  of  new  acquisition  procedures.  It  also  acquires  and 
adapts  tools,  and  disseminates  guidance  that  can  facilitate  new  and 
improved  procedures. 

Project  5720  collects  and  analyzes  lessons  learned  and 
engineering  data  from  ESD  programs  in  order  to  help  plan  improved 
software  acquisition  strategies,  estimate  software  development 
efforts  more  easily,  and  assess  the  status  of  a  software 
development • 

Staff  on  Project  5720  are  a  skilled  resource  for  short-term  and 
long-term  support  to  ESD  programs.  Project  5720  also  provides 
computer  facilities  and  tools  that  can  be  used  to  directly  support 
software  engineering  and  software  acquisition  management  activities 
of  programs  and  to  demonstrate  new  software  engineering  tools  and 
techniques . 

Project  5720  staff  coordinate  activities  with  ESD  management, 
the  Software  Engineering  Institute,  PE  64740F  which  is  the  Air  Force 
software  engineering  technology  application  program  element,  RADC, 
and  other  DoD  organizations.  Under  Project  5720,  some  direct 
support  has  been  provided  to  PE  64740F,  the  Ada*  Joint  Program 
Office,  and  the  DoD  STARS  program,  which  manages  software 
engineering  technology  transition  activities.  Through  these 


*Ada  is  a  registered  trademark  of  the  Department  of  Defense. 
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relationships  to  other  organizations,  Project  5720  staff  maintain 
awareness  of  the  state  of  technology  and  practice,  and  influence  the 
direction  of  policy  and  technology  to  better  meet  the  needs  of  ESD 
programs . 


1.4.4  Technology  Transition:  Project  5720 

Technology  transition  is  the  process  of  changing  the 
state-of-prac t i ce  to  use  new  technology.  Project  5720  performs  this 
important  function  for  software  engineering  technology.  Project 
staff  actively  assess  the  readiness  of  technology  for  use  in  ESD 
programs  and  the  level  of  risk  to  individual  programs,  find 
appropriate  opportunities  to  introduce  the  technology  at  reasonable 
risk,  and  assist  programs  in  the  application  of  the  technology. 

This  activity  often  involves  prototype  applications  of  new 
techniques  and  tools  in  ESD  programs  to  demonstrate  technical 
feasibility  for  that  program.  A  small  portion  of  the  effort  is 
reserved  for  experimentation  in  preparation  for  new  technology 
appl icat ions . 
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SECTION  2 


PROJECT  5720  FY86  ACCOMPLISHMENTS 


2.0  INTRODUCTION 

In  this  section,  the  accomplishments  of  Project  5720  during 
FY86  are  summarized.  Included  are  activities  that  were  initiated 
under  Project  5720  funding  and  subsequently  funded  by  specific 
programs,  and  activities  for  which  other  projects  provided  some  of 
the  funding.  Such  cost  sharing  with  ESD  programs  is  a  part  of  the 
technology  transition  and  acquisition  support  philosophy  of  the 
project . 


2.1  FY86  STAFFING  AND  FUNDING 

About  25  staff-years  were  spent  on  Project  5720  in  FY86.  ESD 
funded  20  staff-years  while  the  remainder  of  the  funds  came  from  Air 
Force  PE  64740F  and  the  Ada  Joint  Program  Office.  These  25 
staff-years  represent  at  least  twice  as  many  individuals  who  spent 
part  of  their  time  on  Project  5720  and  also  worked  on  other 
projects,  where  they  supported  software  acquisition  and  performed 
research  and  experimentation.  About  forty  percent  of  these  were 
management-level  people,  who  participated  in  independent  reviews  of 
programs  and  performed  planning  and  quality  assurance  functions  for 
the  Software  Center. 


2.2  GENERAL  ACQUISITION  SUPPORT 

Tasks  in  this  area  affect  the  practices  used  in  software 
acquisition  management.  Their  purpose  is  to  reduce  acquisition 
risk. 


2.2.1  Policy  Implementation 

The  objective  of  this  set  of  tasks  was  to  recommend  and 
implement  changes  in  software  acquisition  policy  and  practice  at 
ESD. 


In  June  1985  DoD-STD-2167 ,  Defense  Software  Development 
Standard,  was  approved.  Its  purpose  was  to  establish  a  new 
framework  that  is  aimed  at  improving  the  quality  of  delivered 
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software  and  its  maintainability,  and  providing  consistency  among 
the  Services  in  the  procedures  and  the  documentation  for  development 
of  mission-critical  software. 

Under  Project  5720,  an  ESD/MITRE  team  was  assembled  to  help  ESD 
programs  tailor  the  new  standard  for  use  in  their  programs.  Each 
eligible  program* s  RFP  package  was  reviewed  for  an  appropriate 
choice  of  data  requirements.  An  approach  was  used  for  tailoring  the 
standard  that  involved  classifying  the  software  by  how  it  was 
developed  and  delivered  (e.g.,  commercial  or  new),  determining 
appropriate  Data  Item  Descriptions  (DIDs),  tailoring  the  DIDs,  and 
then  tailoring  the  activities,  reviews,  and  products. 

To  acquaint  Program  Offices  and  their  MITRE  support  with  the 
standard,  arrangements  were  made  to  present  courses  on  2167  at  ESD 
and  MITRE.  A  briefing  has  also  been  prepared  to  explain  the  main 
features  of  2167  to  Program  Offices.  The  briefing  compares  2167  to 
prior  standards  and  gives  an  example  of  how  it  can  be  tailored.  It 
will  be  given  to  ESD  Program  Office  staff  in  FY87.  Project  5720 
staff  are  available  to  answer  day-to-day  questions  and  to  review  RFP 
packages  as  well. 

MITRE  worked  with  ESD  in  preparing  an  interim  policy  and 
guidance  letter  on  implementing  2167  that  gives  directions  for 
tailoring  the  Statement  of  Work  (SOW),  Instruction  for  Preparation 
of  Proposals  (IFPP),  System  Specification/Segment  Specification 
(SSS),  and  Contract  Data  Requirements  List  (CDRL)  portions  of  an  RFP 
package  for  conformance  to  2167. 

MITRE  has  been  involved  in  a  workshop  to  review  issues  in 
tailoring  2167  and  has  helped  prepare  proposals  for  modifications  to 
the  standard  or  to  the  text  that  specifies  it. 

In  preparation  for  its  issuance,  MITRE  has  been  gathering 
information  on  DoD-STD-2 168 ,  a  new  software  quality  evaluation 
standard.  Issues  related  to  acquisition  practices  at  ESD  are  being 
identified,  and  a  briefing  is  planned  for  presentation  in  FY87 . 


2.2.2  Software  Reporting  Metrics 

In  FY85,  in  response  to  a  request  from  the  ESD  commander,  MITRE 
defined  a  set  of  Software  Reporting  Metrics  for  use  during  software 
development  on  ESD  programs.  The  metrics  are  a  few,  simple 
quantitative  items  of  data  that  can  monitor  trends  over  time  in  key 
software  cost  drivers  and  can  compare  actual  data  against  plans. 
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They  were  devised  to  provide  more  widespread  and  more  uniform 
visibility  into  the  status  of  software  developments.  These  metrics 
are  intended  to  be  warning  indicators.  They  can  tell  a  Program 
Office  when  to  ask  questions  of  the  managers  in  the  developer’s 
organization  in  order  to  understand  what  is  happening  and  why.  The 
metrics  were  issued  in  FY85. 

In  FY86,  a  revision  to  the  metrics  was  issued  after  review  by 
ESD,  MITRE,  Air  Force,  and  industry  organizations  [ 1 ] •  During  this 
year,  the  project  has  assisted  ESD  programs  in  collecting  metrics 
data.  A  simple  automated  capability  was  assembled  using  a  desktop 
computer  to  analyze  and  plot  metrics  data  recorded  in  a  database  on 
a  larger  computer.  We  have  begun  to  collect  and  analyze  metrics 
data  from  ESD  programs  such  as  the  BMEWS,  MILSTAR,  North  Warning, 
OTH-B,  SPADOC,  and  WIS  programs.  This  will  lead  to  improved  metrics 
and  reporting  procedures. 

To  help  introduce  software  management  reporting,  presentations 
on  the  metrics  were  made  to  staff  of  ESD  Deputies.  The  metrics  were 
tailored  for  programs  on  a  case-by-case  basis  with  the  assistance  of 
Project  5720  staff  who  usually  met  with  SPO  and  contractor  staff. 

In  September  a  tally  showed  that  while  only  one  program  is  reporting 
all  metrics,  at  least  15  were  reporting  some  of  the  metrics  and  2 
were  negotiating  to  add  metrics  reporting  to  their  existing  reports. 
Another  10  programs  were  scheduled  to  start  reporting  metrics  when 
they  sign  contracts. 

The  ESD  Software  Reporting  Metrics  have  become  known  outside 
ESD.  AFSC  has  made  an  adaptation  of  these  metrics  which  has  been 
issued  in  AFSC  Pamphlet  800-43,  "Software  Management  Indicators"  as 
a  common  sense  approach  to  managing  software  acquisitions  [2].  The 
metrics  have  been  briefed  at  the  Defense  Systems  Management  College 
and  conferences.  Papers  are  scheduled  for  the  MILCOM’86  conference 
and  the  National  Joint  Conference  and  Tutorial  on  Developing  Quality 
Software,  both  in  early  October  [3,4]. 


2.2.3  User-System  Interface  Design 

Early  in  FY86,  a  paper  was  published  on  guidelines  for  graphics 
entry  and  display  and  incorporated  into  a  major  revision  of  the  1984 
ESD  report  recommending  guidelines  for  the  design  of  user  interface 
software  [5].  This  document  completes  this  effort,  begun  several 
years  ago  under  PE  64740F  sponsorship.  There  has  been  a  steady 
demand  from  program  offices  and  contractors  for  the  report. 
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2. 2. A  Simulation,  Modeling,  and  Prototyping 


The  work  in  this  area  includes  many  kinds  of  engineering 
activities  that  are  used  to  better  understand  operational  concepts 
and  technical  issues  in  ESD's  systems.  The  results  are  used  in 
defining  system  and  software  requirements,  sizing  the  effort 
required  to  develop  software,  and  dealing  with  perceived  risks 
before  the  final  specifications  are  issued.  These  activities 
provide  a  baseline  for  source  selection  by  increasing  our 
understanding  of  the  key  issues  in  designing  a  system,  and  are  a 
means  for  communicating  with  users  and  developers  concerning  the 
proposed  behavior  of  the  system. 

2.2. A.  1  Performance  Modeling 


Several  years  ago,  PE  6A7A0F  funded  the  development  of  a 
performance  analysis  tool  called  Automated  Interactive  Simulation 
Modeling  System  (AISIM).  This  software  provides  a  graphic  interface 
for  creating  system  models  in  terms  of  the  structure  of  a  system  and 
the  resources  required  by  system  components.  AISIM  executes  a 
discrete  event  simulation  and  prepares  reports  containing  the  times 
and  resource  queueing  and  utilization  statistics  based  on 
statistical  analyses  of  the  results.  Prototype  use  of  AISIM  at  ESD, 
with  MITRE  assistance,  has  led  to  demonstrable  benefits  to  ESD 
programs  in  understanding  performance  issues  of  systems  in  their 
conceptual  phase  as  well  as  during  planned  upgrades.  MITRE  has 
helped  recommend  and  oversee  the  implementation  of  a  series  of 
enhancements  to  AISIM  based  on  our  first-hand  experience  in  its  use. 

In  FY86,  Project  5720  made  AISIM  available  to  ESD  programs, 
provided  assistance  in  using  it,  and  has  developed  some  initial 
models  to  demonstrate  its  utility  to  programs.  With  PE  6A7A0F 
funding,  MITRE  has  also  provided  system  engineering  and  technical 
assistance  in  managing  a  contract  for  implementing  the  latest 
enhancements  to  the  software. 

There  have  been  a  number  of  applications  of  AISIM  in  FY86, 
usually  funded  by  programs.  One  such  application  was  for  the 
proposed  Survivable  Communication  Integration  System  (SCIS).  Prior 
to  source  selection,  a  generic  high-level  model  of  the  SCIS  system 
was  developed  using  information  on  the  type  of  architecture  and 
processors  that  had  been  seen  in  recent  similar  systems.  The  model 
was  exercised  with  a  loading  similar  to  that  in  the  A-level 
specification.  Results  showed  processor  loading  and  throughput  time 
for  messages,  which  provided  insights  that  were  a  basis  for 
discriminating  among  bidders'  proposals  during  source  selection. 
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AISIM  is  being  used  to  analyze  an  upgrade  to  the  MILSTAR 
program  and  also  for  the  UHF  Satellite  Terminal  System  (USTS)  in 
order  to  examine  performance  issues  from  the  A-level  specification. 

2. 2. A. 2  Functional  Simulation 

In  contrast  to  a  performance  simulation,  a  functional 
simulation  models  the  behavior  of  a  system  or  its  software. 
Functional  simulations  can  vary  in  realism  depending  on  the  purpose 
of  the  simulation  and  how  much  is  known  about  the  functional 
requirements . 

For  a  number  of  years,  Project  5720  has  supported  a  Command  and 
Control  Concept  Evaluation  Capability  (CONCAP),  which  consists  of 
equipment,  software  tools,  people  and  services  to  facilitate  the 
rapid  development  of  prototypes  for  portions  of  C^I  systems.  CONCAP 
resources  have  been  used  to  initiate  prototyping  activities  in 
support  of  specific  ESD  programs.  If  the  prototyping  is  extensive 
and  long-term,  CONCAP  provides  access  to  facilities  and  services 
that  can  demonstrate  the  feasibility  of  prototyping  and  help  to 
determine  what  are  appropriate  hardware  and  software  resources  for  a 
separate  prototyping  facility  dedicated  to  a  particular  program.  In 
FY87,  CONCAP  will  be  merged  into  a  MITRE-wide  laboratory  of 
prototyping  and  modeling  facilities. 

In  FY86,  CONCAP  facilities,  tools,  and  services  were  heavily 
used  to  develop  a  series  of  simulations  for  the  Space  Defense 
Initiative  (SDI)  program.  Key  components  of  the  simulation  use 
CONCAP  graphics  software  tools.  Experience  with  CONCAP  has  led  to 
the  definition  and  planned  acquisition  of  more  extensive  facilities 
and  tools  specifically  for  SDI.  That  same  experience,  and  some  of 
the  software  from  the  SDI  simulation,  were  used  to  generate  an 
initial  Air  Defense  Initiative  (ADI)  Simulation  within  a  month. 

With  CONCAP  tools,  a  prototype  system  that  facilitates  the 
handling  of  data  calls  for  program  funding  information  was  completed 
for  ESD/XR.  CONCAP  was  used  for  the  Sentinel  Bright  project  to 
implement  representative  display  software  in  order  to  derive  size 
estimates.  For  the  MILSTAR  program,  a  terminal  prototype  is  being 
developed  to  examine  user  interface  issues.  The  prototype  has  been 
demonstrated  with  various  scenarios  to  the  contractor  as  well. 

Under  Project  5720,  functional  simulations  of  software  designs 
were  also  developed  this  year.  From  the  A-level  specification  and 
the  initial  software  design,  a  model  of  the  interactions  of  the 
Computer  Program  Components  (CPCs)  in  the  software  architecture  of 
the  Joint  STARS  system  was  created.  Its  purpose  was  to  better 
understand  the  nature  of  the  interfaces,  the  data'’ and  control  flows 
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of  significance,  and  the  response  of  the  system  to  various  external 
events.  Useful  information  was  derived  concerning  the  software 
architecture  from  the  modeling  effort,  which  was  programmed  in  Ada. 


2.2.4. 3  Simulation  and  Modeling  Day 

To  encourage  the  use  of  simulation  and  modeling  and  to 
facilitate  technical  interchanges,  Project  5720  arranged  a 
Simulation  and  Modeling  Day.  A  series  of  MITRE  speakers  presented 
their  approaches  to  using  prototyping  and  simulation  and  their 
impact  on  the  ESD  programs.  The  conclusions  of  the  meeting  were 
that  we  are  using  modeling  and  simulation  techniques  and  important 
decisions  are  based  on  the  results.  We  tend  to  select  tools  that 
are  readily  available,  rather  than  finding  the  optimum  tool.  We  are 
also  making  progress  in  applying  knowledge-based  tools  to 
simulation,  modeling,  and  prototyping. 


2. 2. 4. 4  Display  Rapid  Prototyping  System 


PE  64740F  has  identified  a  requirement  for  a  Display  Rapid 
Prototyping  System  (DRPS)  that  can  support  definition  of  user 
requirements  for  Systems.  DRPS  will  permit  the  rapid  development 
of  simulations  of  the  operation  of  a  system,  its  timing,  and  designs 
for  the  user-system  interface.  Two  contractors  have  been  chosen  to 
compete  in  a  study  leading  to  specifications  for  a  new  tool.  MITRE 
will  provide  systems  engineering  and  assistance  to  ESD  in  contract 
management . 


2.2.5  Information  Collection  and  Dissemination 


Project  5720  is  responsible  for  collecting  information  relevant 
to  software  acquisition  and  for  providing  that  information  to  those 
at  ESD  who  can  use  it.  This  service  takes  several  forms. 

2.2.5. 1  Helpl ine 

The  project  maintains  a  Helpline,  a  single  telephone  number  for 
receiving  answers  to  software  acquisition  questions.  The  Helpline 
is  backed  up  by  a  database  of  contacts  for  specific  subjects  and  an 
indexed  log  of  prior  questions. 

2. 2. 5. 2  Newsletter 


Project  5720  publishes  an  internal  quarterly  newsletter  with 
short  summaries  of  software  acquisition  activities  and 
accomplishments  that  might  be  useful  to  the  ESD/MITRE  community. 
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Each  article  gives  a  point  of  contact.  Our  experience  has  shown 
that  many  of  the  articles  lead  to  inquiries  and  further  exchanges  of 
information. 

2. 2. 5. 3  Software  Acquisition  Library 

In  FY86,  we  initiated  a  collection  of  software  acquisition 
documents  from  the  private  holdings  of  MITRE  staff.  These  documents 
can  provide  valuable  guidance  but  have  been  difficult  to  access. 

They  include  examples  of  good  products  from  prior  software 
acquisitions,  or  guidance  produced  within  a  project  or  in  some  other 
organization.  The  collection  has  been  catalogued  and  filed.  A 
computer  allows  browsing  through  the  index.  In  the  next  fiscal 
year,  it  will  be  more  closely  integrated  with  other  online  library 
services . 

2. 2. 5. 4  Courses,  Seminars,  and  Symposia 

In  conjunction  with  ESD  and  with  the  MITRE  Institute,  courses 
related  to  software  acquisition  have  been  offered  locally  in  FY86. 
Project  5720  staff  have  helped  to  plan  the  contents  of  the  courses, 
to  evaluate  potential  speakers,  and  to  teach  the  courses  in  some 
cases.  Over  160  students  attended  courses  on  DoD-STD-2 167 .  At 
least  another  100  students  attended  other  courses  related  to 
software  acquisition. 

At  the  start  of  FY86,  two  courses  organized  and  taught  by 
Project  5720  were  completed.  One  was  "Software  Acquisition: 

Policy,  Terminology,  and  Standards."  The  other  was  "introduction  to 
Ada  and  Software  Engineering."  A  course  entitled  "Software 
Engineering:  Programming  Support  Environments,"  was  offered  at  the 

MITRE  Institute  in  March. 

Ada  was  the  subject  of  several  courses.  About  35  people 
attended  an  "introduction  to  Ada"  programming  course  in  June, 
taught  by  Professor  R.  Vidale  of  Boston  University.  A  one-week 
intensive  course,  "Design  for  Ada  Software,"  by  Mr.  Ed  Berard  was 
arranged  with  the  MITRE  Institute  at  the  request  of  MITRE  people  for 
August  because  the  earlier  course  had  been  oversubscribed. 

An  advanced  workshop/seminar  on  Ada  issues  was  conducted  by 
Professor  Vidale  and  MITRE  oersonnel.  The  course  dealt  with  design 
methodologies,  run-time  performance  issues,  Ada  and  DOD-STD-2 167 , 

Ada  issues  in  security,  and  Ada  as  a  PDL.  The  materials  prepared 
for  this  seminar  will  be  used  as  a  basis  for  the  development  and 
dissemination  of  a  set  of  recommended  practices  for  Ada  software 
acqui s it ion . 
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In  FY87,  courses  are  pLanned  in  Software  Acquisition:  Pol  icy, 
Terminology,  and  Standards;  and  Programming  in  Ada. 

2. 2. 5. 5  Technical  Interchange  Meetings 

Technical  interchange  meetings  have  been  arranged  within  MITRE 
on  particular  subjects.  In  FY86,  these  have  included  Simulation  and 
Modeling,  Ada,  and  Artificial  Intelligence  Applications.  The  AI 
Applications  meetings  have  been  scheduled  on  a  monthly  basis.  These 
meetings  allow  us  to  share  experiences  in  applying  expert  systems  on 
programs  throughout  ESD  and  at  MITRE's  Washington  operations,  and  to 
establish  connections  for  further  exchange  of  information. 

Other  technical  interchanges  have  been  arranged  with  industry 
organizations  to  learn  about  their  software  engineering  research 
activities  and  to  review  recent  products.  These  meetings  are 
coordinated  with  ESD,  and  Air  Force  representatives  attend. 

2. 2. 5. 6  ESD/MITRE  Software  Acquisition  Symposium 

Successful  acquisition  of  software  in  ESD  systems  must  be  a 
joint  effort  of  the  end  user  organization,  the  Program  Office,  and 
the  contractors  who  develop  the  software.  In  order  to  involve 
industry  in  proposing  and  implementing  improvements  in  software 
acquisition  practices  at  ESD,  a  Software  Acquisition  Symposium  was 
organized  under  Project  5720  in  May  1986.  Key  leaders  from  ESD 
software  contractor  organizations  and  ESD  managers  met  to  discuss 
recent  experiences  in  software  acquisition,  to  examine  major  causes 
of  difficulties,  and  to  suggest  improvements  in  Government  and 
industry  policies  and  practices  for  software  acquisition  at  ESD.  A 
compendium  of  the  visual  presentations  was  issued  to  attendees  [6]. 
For  the  ESD/MITRE  community,  videotapes  of  the  sessions  are 
available.  Proceedings  have  been  published  since  the  end  of  the 
FY86  [7]. 

Among  the  recommendations  of  the  Symposium  were  the  following: 

Modify  the  acquisition  strategy  for  C  systems. 

The  focus  was  on  more  prototyping  and  engineering  studies  prior 
to  initial  system  design.  Both  the  user  and  developers  should 
participate  in  this  activity.  This  would  result  in  a  better 
understanding  by  all  parties  of  the  requirements  of  the  system  and 
the  software,  and  would  provide  a  better  basis  for  sizing  and 
costing  the  software.  We  need  to  allow  adequate  time  prior  to 
commitment  to  full  scale  development  for  these  activities. 
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•  3 

Because  of  the  changing  nature  of  C  system  requirements,  most 
developers  endorsed  an  evolutionary  or  incremental  acquisition  with 
early  delivery  of  well-established  capabilities  and  user  feedback 
alternating  with  definition  and  delivery  of  new  or  changed 
capabilities.  Everyone  agreed  on  the  importance  of  firm 
requirements  and  design  at  the  time  of  Preliminary  Design  Review  for 
each  system  increment. 

A  successful  acquisition  strategy  must  also  consider  business 
practices  such  as  contracting.  Industry  leaders  pointed  out  the 
difficulty  of  bidding  on  a  fixed-price  contract  for  Full  Scale 
Engineering  Development  when  little  is  known  about  the  software 
requirements.  A  multi-phase  effort  could  allow  sufficient  analysis 
to  be  performed  to  increase  the  certainty  in  the  cost  estimate  prior 
to  awarding  a  fixed-price  contract.  Contractors  also  cautioned  that 
any  incremental  or  multi-phase  contracting  approach  can  be  damaged 
if  there  are  gaps  in  funding  that  prohibit  the  contractor  from 
keeping  the  software  team  together.  A  method  of  bridging  the  funding 
gaps  is  needed. 

Stronger  management  oversight  and  more  disciplined 
procedures  should  be  used. 

There  was  a  general  endorsement  by  industry  of  the  ESD  Software 
Reporting  Metrics  and  of  both  internal  contractor  and  government 
audits  of  software  status.  Some  speakers  felt  that  DoD-STD-2167 
would  provide  a  more  disciplined  approach  to  software  development 
and  greater  visibility  to  managers  of  the  status  and  quality  of  the 
effort.  Many  contractors  are  developing  internal  standards  that  are 
compatible  with  2167.  The  cautionary  note  on  DoD-STD-2167  was  to 
change  the  strategy,  as  described  above,  to  include  more  prototyping 
and  risk  management. 

For  large  productivity  and  quality  improvements  in 
software  engineering,  integrated  programming  support 
environments  should  be  used. 

Several  speakers  acknowledged  a  shortage  of  software  engineers 
to  develop  large,  complex  systems.  Suggested  remedies  included 
development  of  programming  support  environments  with  workstations 
and  tools  to  support  software  development  activities  and  project 
management.  These  environments  would  serve  to  enforce  standard 
software  development  procedures  and  hopefully  would  improve 
productivity.  A  number  of  contractors  described  internal  efforts  to 
provide  environments.  In  the  near  term,  they  automate  more  routine 
functions  such  as  documentation  generation  and  configuration 
management.  Over  the  longer  term,  larger  gains  in  productivity 
might  be  achieved  by  increased  reuse  of  software  and  by  expert 
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systems  and  knowledge  bases  to  support  the  process  of  software 
development  and  maintenance. 

A  practical  issue  in  the  use  of  programming  support 
environments  is  DOD  policy  on  data  rights:  who  owns  software  used  to 
develop  DOD  systems.  The  results  of  a  Software  Engineering 
Institute  study  on  DOD  data  rights  policy  were  presented.  It  was 
noted  that  the  current  laws  concerning  software  rights  are  confusing 
at  best.  Some  revisions  may  be  needed  to  provide  economic 
incentives  for  industry  to  develop  programming  support  environments 
and  use  them  on  DOD  programs,  and  for  industry  to  reuse  application 
software  as  well. 

-  The  Ada  language  and  programming  support  environments  are 
beneficial  to  CJ  systems  in  the  long  run  and  should  be 
used  now. 

Speakers  indicated  that  programs  that  are  being  developed  now 
for  long-term  operation  must  use  Ada,  since  other  current  languages 
will  be  increasingly  less  effective  in  meeting  the  demanding 
requirements  of  these  systems  over  the  next  20  years. 

Ada  must  be  introduced  in  measured  steps,  which  control  risk 
and  allow  vendors  to  increase  the  capabilities  of  compilers  and 
other  tools.  Speakers  recommended  the  following:  Ada  as  a  Program 
Design  Language  now;  testing  of  available  Ada  tools  and  environments 
for  specific  applications;  training  for  both  developers  and  managers 
(in  industry  and  in  government);  prototype  applications  of  Ada;  and 
more  focus  on  real-time  applications,  distributed  systems,  and 
fault-tolerant  features  in  new  Ada  compilers. 

A  final  recommendation  was  made  that  we  look  more  closely  at 
the  effect  of  Ada  on  software  maintenance  before  we  begin  to  field 
Ada  software. 


2.3  TECHNOLOGY  TRANSITION 

Recent  studies  have  shown  that  the  transition  of  software 
technology  from  an  experimental  state  to  common  practice  takes  as 
long  as  17  years.  Project  3720  has  a  strong  responsibility  to 
accelerate  the  technology  transition  process  for  software 
engineering  without  causing  undue  risk  to  ESD  programs. 

Technology  transition  requires  knowledge  of  the  readiness  of 
technology  for  use  and  opportunities  to  apply  it.  The  best  vehicle 
we  have  found  for  accomplishing  that  transition  is  people. 
Therefore,  Project  5720  provides  the  knowledge  of  technology  and 
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encourages  people  to  move  from  the  project  to  acquisition  programs 
with  that  knowledge.  In  addition,  Project  5720  broadcasts 
information  to  the  ESD/MITRE  community  through  briefings,  seminars, 
and  publications.  Where  deficiencies  are  found  in  the  direction  of 
current  technology  for  ESD  requirements,  Project  5720  staff  take 
steps  to  influence  efforts  of  other  organizations,  and  to  look  for 
solutions  themselves. 

The  technology  transition  efforts  in  Ada,  Artificial 
Intelligence  applications,  and  Computer  Security  for  FY86  are 
described  below.  The  latter  two  areas  of  technology  transition  were 
supported  by  AF  PE  64740F,  Computer  Resource  Management  Technology. 


2.3.1  Introduction  of  Ada  at  ESD 


The  goal  of  the  Ada  introduction  task  is  to  facilitate  the 
successful  introduction  of  Ada  into  ESD  programs  through  acquisition 
support,  technology  application,  training,  and  participation  in  the 
government  and  industry  Ada  community. 

2.3.1. 1  Acquisition  Support 

Project  5720  staff  have  provided  consultation  and  short-term 
support  to  over  10  ESD  programs  in  dealing  with  Ada-related  issues 
such  as  RFP  preparation,  source  selection,  and  contractor  product 
review.  Staff  have  also  participated  in  ESD  and  MITRE  review  teams, 
and  have  worked  with  contractors  to  help  solve  problems  with  Ada 
performance  during  acquisition. 

The  Software  Center  and  Project  5720  have  provided  assistance 
to  an  ESD  program  in  developing  an  innovative  technique  for 
evaluating  bidders'  software  development  methodology  and  use  of  an 
Ada  Design  Language.  During  the  Full  Scale  Development  source 
selection,  each  bidder  will  be  given  a  software  engineering  exercise 
in  which  he  applies  his  software  development  methodology  on  a  sample 
problem.  The  objective  is  to  illustrate  the  bidders'  design 
approach  and  interim  products  that  use  Ada  for  design.  Project  5720 
staff  are  working  jointly  with  the  Program  Office  staff  to  do  a 
practice  application  of  the  exercise  in  order  to  refine  it,  to 
anticipate  issues  that  may  arise  during  bidders'  use  of  it,  and  to 
produce  objective  source  selection  criteria  to  be  used  in  evaluating 
the  bidders'  performance. 

While  the  programs  usually  bear  much  of  the  cost  of  such 
activities,  Project  5720  assures  that  people  with  the  necessary 
skills  and  knowledge  are  available  and  verifies  the  feasibility  of 
such  recommendations  if  necessary. 
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2 . 3 . 1 . 2  SAC  Ada  Development  and  Maintenance  Study 


Project  5720  funded  a  study  to  help  develop  an  approach  to  Ada 
software  development  and  maintenance  at  SAC.  The  study  surveyed 
selected  SAC  systems  to  identify  candidates  for  an  Ada  transition 
study.  After  surveying  relevant  software  technologies,  the 
components  of  a  software  development  and  maintenance  environment 
were  recommended  to  meet  the  needs  of  SAC  directorates.  In  the 
final  phase,  completed  in  mid“FY86,  a  transition  plan  was  proposed 
for  a  specific  set  of  SAC  systems  to  adopt  an  Ada  environment  and  to 
train  for  its  use.  The  study  advocates  use  of  a  standardized  Ada 
software  development  and  maintenance  environment. 

2.3. 1.3  Technology  Application 

Firsthand  application  of  Ada  technology  in  the  context  of  ESD 
system  requirements  and  ESD  policies  and  practices  allows  MITRE  to 
experience  the  strengths  of  Ada  and  to  devise  solutions  to  problems 
which  may  arise  in  the  use  of  Ada  at  ESD. 

2.3. 1.3.1  An  Application  of  Ada  Design  Methods 

In  FY85,  an  investigation  was  begun  into  how  Ada  could  be  used 
as  a  Program  Design  Language  (PDL)  along  with  formal  software 
development  methods.  The  investigation  took  the  form  of  a  model 
design  exercise  called  MIMSY.  A  top-level  design  was  developed  and 
documented  in  accordance  with  procedures  and  documentation  specified 
in  DoD-STD-2167.  Two  versions  of  the  Software  Top-Level  Design 
Document  were  produced.  Each  was  about  40  pages  long,  with  85%  of 
each  being  Ada  PDL. 

To  enhance  the  realism  of  the  design  exercise,  personnel  were 
assigned  roles  corresponding  to  those  of  the  typical  organizations 
involved  in  software  acquisition,  and  formal  reviews  were  were 
conducted  through  a  second  Preliminary  Design  Review. 

A  combination  of  existing  design  methods,  formal  design 
languages  and  design  notations  were  selected  from  current 
technology.  The  results  are  documented  in  an  ESD  Technical  Report 
[8].  The  results  were  also  presented  at  a  recent  SIGAda  conference. 
The  investigation  confirmed  the  usefulness  of  formal  design  methods 
and  notations  with  Ada,  provided  developers  and  reviewers  are 
familiar  with  the  techniques.  A  tasking  model  of  system  behavior 
was  also  recommended. 
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2. 3. 1.3. 2  ESP  Acquisition  Support  Environment  (EASE) 


In  preparing  for  the  use  of  Ada  in  ESD  programs,  we  realize  the 
need  for  tools  that  will  more  effectively  support  the  systems 
engineer's  role  of  analyzing,  evaluating,  and  managing  Ada  products 
of  software  developers.  Under  Project  3720,  we  have  begun  to 
provide  an  open,  extensible  architecture  for  integrating 
independently  developed  tools  with  capabilities  for  analyzing  Ada 
software.  This  environment  is  called  EASE. 

In  FY86,  the  initial  framework  was  designed  and  is  being 
developed  for  rapidly  integrating  existing  Ada-based  tools.  Many 
tools  that  run  under  Unix*  including  editors,  testing  tools,  and 
formatters  can  be  incorporated  now.  Building  on  existing 
capabilities  in  a  Sun**  workstation,  we  have  added  a  set  of  process 
control  primitives  to  make  it  easier  to  integrate  tools.  In  FY87, 
specification  and  implementation  of  a  common  data  representation 
will  be  initiated  to  integrate  data  flow  for  all  EASE  tools. 

Because  of  the  similarities  between  programming  support 
environments  and  EASE,  this  effort  provides  us  with  insights  into 
the  status  of  environments  for  developing  Ada  software  and  will 
result  in  useful  tools  for  analyzing  Ada  products  in  software 
acquisitions . 

2.3. 1.3.3  Ada  Functional  Simulation 


In  Section  2. 2. 4.1,  the  use  of  functional  simulations  to 
analyze  software  designs  was  described.  Three  such  models  were 
developed  in  Ada  for  ESD  programs.  The  model  developments  had  a 
dual  purpose:  to  provide  analyses  of  designs,  and  to  learn  how  Ada 
can  be  used  for  high-level  simulations.  The  conclusions  concerning 
Ada  were  that  some  difficulties  existed  in  using  a  strongly-typed 
language  to  represent  abstract,  high-level  designs,  and  some 
problems  with  representing  timing.  The  tasking  model  of  the  Ada 
language  was  suited  for  behavioral  models. 


*  Unix  is  a  registered  trademark  of  AT&T. 

**  Sun  is  a  registered  trademark  of  Sun  Microsystems. 
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2*3*2  Ada  Joint  Program  Office  Support 


A  small  amount  of  support  has  been  given  to  the  Ada  Joint 
Program  Office  (AJPO)  in  specification  of  the  Common  Ada  Programming 
Support  Environment  (APSE)  Interface  Set  (CAIS).  The  CAIS  is  a 
standard  operating  system  interface.  Its  purpose  is  to  allow  the 
interchange  of  Ada  programming  tools  and  project  data  among 
environments  on  different  computers  and  under  different  operating 
systems . 


2.3.2. 1  Prototype  CAIS 

The  objective  of  this  task  is  to  build  a  prototype  CAIS  by 
implementing  a  full  operating  system  in  Ada  for  the  Intel  80386 
microprocessor.  This  prototype  will  provide  important  feedback 
about  the  impl ementabil i ty  and  performance  characteristics  of  the 
CAIS  to  its  developers  before  standardization.  In  FY86 ,  parts  of 
the  design  were  completed,  including  the  lowest  levels  closest  to 
the  hardware,  a  virtual  memory  mechanism  and  part  of  the  task 
scheduler . 

2. 3. 2. 2  CAIS  Consultation 


In  the  past  two  years  with  the  sponsorship  of  other  projects, 
MITRE  has  implemented  the  CAIS  under  several  operating  systems  and 
has  extended  its  design  to  support  distributed  processing  over  local 
area  networks.  A  small  task  under  Project  5720  this  year  has 
provided  the  Ada  Joint  Program  Office  and  other  organizations  with 
the  expertise  gained  while  MITRE  performed  this  research. 
Specifically,  MITRE  has  provided  answers  to  technical  questions  and 
provided  advice  to  the  Kernel  Ada  Programming  Support  Environment 
(KAPSE)  Interface  Team  (KIT),  the  KAPSE  Interface  Team  from  Industry 
and  Academia  (KITIA),  the  CAIS  Working  Group  (CAISWG),  and  the  AJPO. 
MITRE  has  also  provided  written  comments  on  specific  technical 
issues  and  reviews  of  the  proposed  Military  Standard  CAIS 
specification . 

2.3.3  Ada  Community  Participation 

Under  Project  5720,  MITRE  has  actively  participated  in 
organizations  that  develop  and  influence  Ada  technology  and  policy. 
This  has  allowed  us  to  maintain  current  information  about 
technology,  policy,  and  experience,  and  to  communicate  the 
experiences  and  concerns  of  ESD  to  those  organizations. 

MITRE  staff  are  permanent  members  of  the  KIT/KITIA  and  the  APSE 
Evaluation  and  Validation  Teams.  We  have  also  participated  in  other 
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workshops  and  professional  group  meetings.  A  talk  was  delivered  at 
a  recent  SIGAda  meeting  on  the  our  Ada  work. 


2. 3. A  Artificial  Intelligence  Applications 


The  purpose  of  these  tasks  to  to  help  ESD  programs  to 
understand  and  use  AI  technology.  The  approach  has  been  to  obtain 
and  use  representative  state-of-the-art  hardware  and  software  for 
applications  in  which  the  users  are  an  ESD  Program  Office  and  a 
MITRE  systems  engineer.  These  prototypes  show  how  well  the 
technology  can  be  used,  the  effort  and  skills  required  for  its  use, 
and  the  process  to  acquire  expert  systems.  All  of  this  information 
is  needed  to  advise  ESD  programs  on  the  application  of  expert 
systems,  and  may  result  in  tools  that  improve  system  and  software 
acquisition  at  ESD. 

The  work  in  AI  applications  was  partly  supported  by  PE  64740F, 
and  influenced  by  other  AI  research  and  experimentation  at  MITRE, 
under  Air  Force  and  NASA  sponsorship. 

2.3.4. 1  PE  6474QF  Planning 

PE  64740F  has  responsibility  for  transitioning  technology  for 
the  management  of  software  acquisitions.  MITRE  is  providing  the 
program  with  a  survey  of  the  status  of  AI  research  and  development 
work  that  might  be  applied  to  projects  within  this  program  element. 
A  number  of  specific  tasks  have  been  identified  for  the  program  to 
sponsor . 


2. 3. 4. 2  System  Sizing  and  Costing  Environment 

The  objective  of  this  task  is  to  develop  a  knowledge  base  and 
automated  aids  capable  of  helping  an  analyst  to  relate  the 
functional  requirements  of  a  C^I  system  to  the  level  of  effort 
required  to  build  the  system.  This  is  a  technology  prototype  which 
uses  expert  system  technology  to  aid  in  software  project  management. 
For  most  software  cost  models,  the  accuracy  of  the  estimate  depends 
on  how  well  the  analyst  can  estimate  the  level  of  effort  required  to 
develop  software,  which  in  turn  depends  on  estimates  of  its  size  and 
complexity.  It  is  difficult  to  make  such  estimates  from  the 
functional  requirements  alone.  This  capability  will  provide  models 
of  typical  systems  to  be  used  in  cost  estimation  from 
specifications  before  designs  exist.  Sizing  information  related  to 
functional  requirements  will  also  be  included  in  the  knowledge  base. 
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In  FY86,  a  demonstration  has  been  prepared  to  illustrate  types 
of  capabilities  that  might  be  provided  and  to  AI  techniques  for 
achieving  them.  During  the  latter  half  of  FY86,  this  work  was 
funded  by  PE  64740F  and  will  continue  into  FY87. 


2 . 3 . 4 . 3  An  Expert  System  for  Modeling  and  Analysis  of  Requirements 
and  Design 

As  part  of  the  AI  Applications  technology  transition 
activities,  another  early  prototype  has  been  developed  that  uses  an 
expert  system  building  tool  ( Intel 1 icorp 1 s  Knowledge  Engineering 
Environment  (KEE))  for  its  implementation.  This  enables  MITRE  to 
demonstrate  and  assess  the  impact  of  such  tools  on  the  acquisition 
process  for  expert  systems. 


The  prototype  will  support  the  generation  of  requirements 
specifications  and  their  representation  as  text,  graphic  schematics, 
and  models  that  can  be  analyzed.  It  will  contain  a  knowledge  base 
of  typical  CJ  system  requirements  and  system  designs  so  a  user  can 
build  a  specification  from  generic  examples  of  similar  systems. 
Representations  will  be  available  for  expressing  functions  and  their 
relationships,  interfaces,  and  expected  behavior.  Requirements  and 
designs  will  be  analyzed  for  completeness  and  consistency, 
individually  and  between  them.  Dynamic  analyses  would  allow  the 
system  behavior  to  be  observed  in  response  to  simulated  inputs,  and 
performance  information  to  be  collected. 


This  year,  an  initial  prototype  has  been  demonstrated  to 
illustrate  capabilities  that  seem  useful  and  feasible.  A  framework 
has  been  designed  for  implementing  capabilities.  The  results  have 
been  described  for  several  technical  workshops. 


2.3.5  Computer  Security 

MITRE  has  performed  a  number  of  tasks  in  Project  5720  for  the 
Computer  Security  program  of  PE  64740F.  The  objective  of  the 
Computer  Security  Applications  task  is  to  help  ESD  and  the  Air  Force 
Computer  Security  Program  Office  (AFCSPO)  to  transition  computer 
security  technology  into  the  Air  Force.  There  were  two  efforts 
under  the  project:  risk  analysis  technology  support  and  expert 
systems  for  authentication. 

2.3. 5.1  Automated  Threat  Analysis  Methodology 

Air  Force  regulations  mandate  periodic  risk  analysis  of  ADP 
systems,  facilities,  and  installations,  as  part  of  an  overall 
security  program.  MITRE  provided  both  a  conceptual  framework  for 
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risk  analysis  of  an  ADP  facility  and  specific  guidelines  for  risk 
analysis  tasks  based  on  facility  size,  complexity,  and  data 
sensitivity.  It  included  forms  for  gathering  information  and 
examples  of  threats  and  countermeasures  to  aid  the  risk  analyst, 

MITRE  has  given  systems  engineering  support  to  ESD  for 
monitoring  a  contract  to  develop  a  production-quality  automated  risk 
analysis  tool  based  on  the  prototype  Automated  Threat  Assessment 
Methodology  (ATAM)  developed  at  MITRE  in  FY84  -  FY85,  During  FY86, 
the  prototype  was  demonstrated  at  SAC, 

2, 3,5,2  User  Authentication 


The  objective  of  this  task  is  to  apply  current  artificial 
intelligence  technology  to  the  problems  of  authenticating  a  user  of 
a  computer  system.  During  FY86,  a  literature  search  was  conducted 
and  a  model  proposed  which  involved  asking  the  user  questions  from 
his/her  own  experience.  During  a  registration  phase,  a  valid  user 
enters  a  set  of  questions  and  acceptable  answers.  These  questions 
are  then  selected  for  use  at  logon  to  authenticate  the  user.  After 
experimentation,  a  prototype  was  developed.  Other  techniques  are 
being  explored  in  addition  to  this  solution, 

2  •  3  •  5 • 3  Secure  Local  Area  Network  (LAN) 


ESD  has  a  contract  with  Verdix  Corporation  under  the  Small 
Business  Innovative  Research  program  which  has  resulted  in  the 
design  and  development  of  a  prototype  of  the  Verdix  Secure  Local 
Area  Network,  which  provides  multilevel  secure  datagram-level 
service  to  LAN  nodes.  In  the  next  phase,  Verdix  will  install  the 
prototype  and  evaluate  it  in  a  specific  Air  Force  application, 

MITRE  is  contract  monitor  and  will  assist  in  selection  of  the  user 
site,  and  reviewing  contractor  progress.  This  work  began  late  in 
the  fiscal  year  with  a  review  of  the  Statement  of  Work  for  the  next 
phase  of  the  contract. 
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